Weighted Generalized LDA for Undersampled Problems

Linear discriminant analysis (LDA) is a classical approach for dimensionality reduction. It aims to maximize between-class scatter and minimize within-class scatter, thus maximize the class discriminant. However, for undersampled problems where the data dimensionality is larger than the sample size, all scatter matrices are singular and the classical LDA encounters computational difficulty. Recently, many LDA extensions have been developed to overcome the singularity, such as, Pseudo-inverse LDA (PILDA), LDA based on generalized singular value decomposition (LDA/GSVD), null space LDA (NLDA) and range space LDA (RSLDA). Moreover, they endure the Fisher criterion that is nonoptimal with respect to classification rate. To remedy this problem, weighted schemes are presented for several LDA extensions in this paper and called them weighted generalized LDA algorithms. Experiments on Yale face database and AT&T face database are performed to test and evaluate effective of the proposed algorithms and affect of weighting functions. Key–Words: Feature extraction; dimensionality reduction; undersampled problem; weighting function; misclassification rate